Disordered Environments in Spatial Games 
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The Prisoner's dilemma is the main game theoretical framework in which the onset and main- 
tainance of cooperation in biological populations is studied. In the spatial version of the model, we 
study the robustness of cooperation in heterogeneous ecosystems in spatial evolutionary games by 
considering site diluted lattices. The main result is that due to disorder, the fraction of coopera- 
tors in the population is enhanced. Moreover, the system presents a dynamical transition at p* , 
separating a region with spatial chaos from one with localized, stable groups of cooperators. 



I. INTRODUCTION 

Game theoretical methods have been applied, in a 
quite sucessfull way, to several different fields [0J|], bi- 
ology being one of the most successful branches M pi. 
In these evolutionary games, rewards are translated in 
terms of subsequent reproductive success, a more natu- 
ral and clear scale than the rationality dependent ones 
used in, for example, economy. Of particular interest is 
the emergence and sustainability of cooperation, what 
has attracted a lot of attention as it poses a difficult 
problem from an evolutionary point of view since co- 
operators, by their nature, have their fitness decreased 
while interacting with defectors. Theories advanced to 
explain the evolution of cooperation usually consider ei- 
ther kin selection ||, reciprocal altruism |7]]|] or group 
selection |] |lj, the separation between them not al- 
ways being clear. Group foraging and young raising J14| , 
alarm calls |15|,|l6[ , bacteria-infecting viruses [jl7| , preda- 
tor inspection by fish |lq| , birds female-female coopera- 
tion |lj|, cleaner fishes ]20|, arc but a few examples of 
such acts in biological populations, ranging from simple 
to complex organisms. A number of other examples may 
be found in ref . || . 

The Prisoner's Dillema game is generally studied as an 
archetypical model for reciprocal altruism. Each of the 
two players either cooperates (C) or defects (D), with- 
out knowledge of the opponent's strategy. The result 
depends on the mutual choice and is given by the pay- 
off matrix whose elements are: a reward R (punishment 
P) if both cooperate (defect), S (sucker's payoff) and T 
(temptation) if one cooperates and the other defects, re- 
spectively. Moreover, these quantities should satisfy the 
inequalities T > R> P > S and 2R>T + S. It is clear 
that, independent of the opponent's choice, defecting is 
always the best bet. Thus, two basic evolutionary prob- 
lems are i) the onset of cooperation, that is, under which 
conditions a given cooperative behavior can invade a pop- 



ulation of defectors and, once established, ii) its stability, 
that is, under which conditions the population of cooper- 
ators is uninvadable. In other words, we are looking for 
Evolutionary Stable Strategies (ESS) dUE^I wnere there 
is stable mutual cooperation between individuals. In a 
random mating, infinite population of asexual (haploid) 
elements, where two pure strategies are present (coop- 
erators C and defectors D), it can be shown that de- 
fecting will be the most rewarding strategy (an ESS). If 
instead of only interacting once or a known finite number 
of times, the players have a probability w > of meeting 
again in the next round and remember the chosen strate- 
gies in previous encounters, more complex rules may be 
devised M. For instance, in a round robin tournament, 
Axelrod [jj found that tit-for-tat (TFT) (starts cooper- 
ating and, after that, do what the opponent did in the 
previous step) was the most successful strategy, but other 
successful strategies have also been found J22fl . Neverthe- 
less, TFT cannot invade a population of defectors before 
reaching a minimum population, what can be achieved 
in several ways ||,|23|,|24| . 

Following a suggestion of Axelrod p), Nowak and 
May p5|-p7[ have shown how, cooperation can arise even 
with only pure, memoryless strategies, like always coop- 
erate or always defect in the presence of spatial struc- 
ture. They considered a deterministic cellular automa- 
ton consisting of a square lattice with near and next 
nearest neighbors interactions and self interaction. Since 
the agents are spatially localized, they are more likely 
to interact only with their nearest neighbors, differently 
from the standard, mean field-like approach, that con- 
sider an infinite, random mixing population. The rea- 
sons for this are manyfold: individuals usually occupy 
well-defined territorial regions, individuals do not move 
far from their places of birth (population viscosity ||), 
interactions occur in places where animals usually meet 
such as water ponds, etc. After the combats, each player 
compares its total payoff with the ones of its neighbors 
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and changes strategy, following the one with the great- 
est payoff among them. For a certain range of values of 
the payoff matrix, very complex spatial patterns show up 
with cooperators and defectors coexisting (spatial chaos). 
In these structured populations, cooperative strategies 
can build clusters in which the benefits of mutual coop- 
eration can outweight losses against defectors, maintain- 
ing the population of cooperators stable. Irrespective 
of the initial state that may be chosen with either only 
one initial defector or a fraction of randomly distributed 
defectors, the asymptotic density depends only on the 
payoff matrix parameters. The actual values depend on 
the neighborhood chosen for the dynamics and whether 
self- interactions are included or not |p5| , p6[ . This coop- 
eration enhancement effect due to the spatial structure 
is also seen in other games. For instance, in the spa- 
tial version of the Hawk-Dove (HD) game |2q , g8Lg9[ , the 
density of doves (analogous to cooperative individuals) is 
increased by the spatial structure, although in this game 
there would exist polymorphism even in the absence of 
spatial effects. In a similar way, several strategies are 
more successful in a spatially structured population than 
otherwise po| , |3l| ] . 

In real populations, however, not all individuals in- 
teract the same number of times either due to the non 
sincronous character of the interaction or to the environ- 
ment structure that prevents some of the contacts. Both 
the spatial and temporal aspects of the environment af- 
fect the interactions between individuals, being central 
issues in ecological and evolutionary theory. Thus, a nat- 
ural question arises: how is cooperation affected by this 
inherent inhomogeneity? In other words, how robust is 
cooperation in the presence of disorder? In this paper we 
try a simple approach to this question by considering a 
regular lattice where some sites are empty. Dilution can 
be either quenched (fixed) or annealed (evolving). In the 
latter case, individuals may diffuse in the lattice, what 
will be considered in detail in a future work. Quenched 
vacancies (or defects) may account for the presence of 
environmental features (e.g., geographical) in the game, 
making some individuals have less neighbors than others. 
Deviations from the ordered lattice can also be achieved 
in several other ways. For instance, by allowing that 
some of the individuals also interact with distant ones, 
in a small- world network fashion p2 33 1. In |2^,Q, a di- 
luted lattice was used with individuals interacting inside 
a region of radius r, and persistent polymorphism of C's 
and .D's was found, unless r was made too big, reach- 
ing the long range connectivity associated to mean field 
behavior, where the defector population dominates. Nev- 
ertheless, their treatment was quite brief and several in- 
teresting dynamical behaviors have passed unnoticed, as 
well as the important issue of whether disorder enhances 
or not the fraction of cooperators in the population. 

Here we show that depending on the amount of dis- 
order, cooperation can be enhanced, there being a point 
where a dynamical transition settles in, separating the re- 
gion with spatial chaos from the one with localized groups 



of cooperators. The paper is organized as follows. In sec- 
tion [n| the diluted version of the spatial prisoner's game 
is pres ente d and the order parameters are defined. In 
section Hl| the main results are presented and, finally, in 
section IV, we present the conclusions and comments. 



II. THE MODEL 

We consider the spatial version of the prisoner's 
dilemma [jl 25 - 27],|3(j,|34| pt\ , placing the interacting el- 
ements in the vertices of a d-dimensional array, usu- 
ally a hypercube, with periodic boundary conditions. 
The results presented here are for d = 2. Differently 
from the original case, we allow that some of the sites 
may be empty. To describe the occupation of the i-site 
(1 < i < N — L d , where L is the system linear dimen- 
sion) we take rii to be either I, if the site is occupied or 0, 
otherwise. In every generation, each individual assumes 
an unchangeable strategy from a set Q and, following 
rcf . |2q] , we only consider the simplest case of pure strate- 
gies, C and D, represented by the variable Si = ±1, re- 
spectively. In each step (generation), the i-th individual 
(n, =/= 0) combats with all other elements inside a given 
neighborhood V;, and accumulates a payoff pi, depending 
on the chosen strategies, accor ding to the reduced payoff 
table for the prisoner's game J25|: R=l, P = S — 
and T = b > 1 , reducing the problem to only one param- 
eter (besides the density). For the case considered here, 
where the neighborhood Vi is restricted to the nearest 
neighbours of site i and no self interactions are included, 
both C and D coexist in the region 4/3 < b < 3/2 in 
which also the number of active sites (see below) is large. 
For I < 6 < 4/3 cooperators dominate while for b > 3/2, 
defectors are dominant. The player's payoff is a mea- 
sure of its reproductive success: when reproducing, the 
i-th. element compares its own payoff with all j s V» and 
changes to the strategy of the site that has the greatest 
payoff in {i} U V,. In this way, the global density is kept 
fixed since no empty site will be ever filled. 

To characterize the macroscopic behavior of the system 
we introduce two order parameters. Let p c (t) represent 
the fraction of cooperators at a given time: 
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where TV = L d is the total number of sites. Clearly, 
Pd(t), the defectors density, is Pd(t) = p — p c (t), where 
p is the total density. Since we are interested in the 
long time regime, and the results depend on the choice of 
the frozen empty sites, we define the order parameter as 
the average over time ((. . .)) and over the realizations of 
the disorder ( 7TT ) of the asymptotic cooperators density, 
Pc = (Pc(oo)), for large N. Sometimes it is more useful to 
have the relative cooperators density, p c /p. Thus, p c = 



means that the population was fully invaded by defec- 
tors and p c = p, by cooperators. An intermediate case, 
< p c < p in which both strategies coexist is also pos- 
sible. Moreover, it is interesting to know the fraction of 
active sites, that is, the fraction of elements that change 
strategy in time: 
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This defines our second order parameter, p a = (p a (oo)}. 
Thus, p a = means that all sites are frozen and p a = p 
that all elements are changing strategy. 



active sites are bounded (pinned) to small regions due to 
the presence of defects, as will be shown below. Below 
the transition, the approach to equilibrium is exponen- 
tially fast, p c (oo) — p c (t) ~ exp(— t/r), r diverging as 
one approaches the point p* . This is because most of the 
interactions occur inside of the localized groups. Also, if 
the lattice is completely filled, the approach is fast. On 
the other hand, just above the critical value, the presence 
of defects, and the fact that only some of the groups are 
depinned makes the approach to equilibrium extremely 
slow, power-law like, what is reminiscent of disordered 
systems. 



III. RESULTS 



In fig. |l|, the asymptotic cooperator density p c /p is 
plotted against the total occupation of the lattice, p. If 
the occupation fraction is near zero, almost all sites are 
isolated and do not change strategy since there is no com- 
bat at all, the asymptotic density being the same as the 
initial one: p c = p c {0) = p/2. Indeed, for small p, the 
curves for different values of b merge. As the density 
increases, the probability of occurring pairs of occupied 
sites increases and, irrespective of the value of b, all CD 
pairs will become DD, what can be seen as a decreas- 
ing curve of p c from the origin. Around p ~ 0.3, as the 
number of interacting individuals increases due to the in- 
creasing clusters sizes, the dynamics will define the fate of 
each cluster and the curves for different values of b depart: 
the higher b is, the better is for defectors and the lower is 
the corresponding cooperator curve. Still further, when 
clusters of cooperators have a reasonable probability to 
be formed, their density start to increase recovering, as 
the total density approaches unity, the Novak-May re- 
sults p5fl . The most interesting case happens if b is in 
the active region, 4/3 < b < 3/2, where the behavior is 
not monotonic and a sharp decrease in p c appears near 
p* ~ 0.95. At the same point, the fraction of active sites 
presents a sharp increase, as can be seen in the inset of 
fig. [y. Notice that for the others regions, the behavior 
is almost uniform, with a very small number of active 
sites. Interestingly enough, depending on the region of 
b, the optimum density that maximizes the fraction of 
cooperators occurs at different values: for b < 4/3, the 
more individuals, the better, and their maximum occurs 
at p = 1; above b = 3/2, on the other hand, the less 
occupied the network, the better, due to the exploitation 
by defectors; the maximum occurs in the limit p — ► 0. In 
these cases, respectively, cooperators and defectors are 
in advantage when interacting. In the intermediate, ac- 
tive region, the behavior is non trivial and the maximum 
happens at the transition point p*. Remarkably, this 
point is much above the site percolation transition, that 
for a square lattice is located at p per c — 0.59 Q: al- 
though there is a connected, infinite cluster, regions of 
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FIG. 1. Asymptotic fraction of cooperators, p c /p, as a 
function of the lattice occupation, p for several values of 
b. The initial state has p/2 cooperators, the lattice size is 
L = 100 and averages are taken over 100 samples. 



In the active region, for different initial concentrations 
of cooperators, it can be seen that almost up to p*, the 
asymptotic density depends on the initial state, as shown 
in fig. @. For small p, since the clusters are independent, 
obviously the higher the initial fraction of cooperators is, 
the higher p c will be. Notice that the optimum density 
for cooperation is a function of the initial density of co- 
operators: for initial concentrations below the height of 
the peak at p* , p c (0) < 0.54p, p* is the ideal total con- 
centration for cooperators, while above, the smaller is p, 
the better. Slightly below p* all curves merge because 
the cluster boundaries are no longer completely pinned 
and the memory of the initial state is washed out. 




spin flips infinitely many times. Since we are near the 
transition, the quantity of pinned sites is quite reduced. 
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FIG. 2. The asymptotic fraction of cooperators for several 
different initial densities of cooperators, L = 100 and b = 1.4. 



Although both cooperators and active sites densities 
are useful parameters to describe the system asymptotic 
behavior, they are not sufficient in order to understand 
the complex dynamical behavior that arises in the pres- 
ence of disorder. To clarify what is going on at the transi- 
tion point, we also measured the persistence, the fraction 
P{t,t w ) of sites that do not change strategy between an 
initial waiting time t w and the time t > t w |39| |42f| , as 
can be seen in figs, ra and 0. Differently from the fraction 
of active sites, the persistence is a very complex measure 
that depends on the whole time history of the system 
since t w . For example, if the persistence does not go 
to zero, we know that there is a fraction of sites that 
flips only finitely many times |^3|,Q (blocking) and do- 
main wall movements are constrained (pinning). That is 
precisely what happens for p < p* , as shown in fig. |3|: 
after an initial decrease, the persistence attains, for large 
times, a plateu whose value depends both on p and t w . 
Denoting this plateu by P(oo, t w ), we notice that it goes 
to zero at p*, as shown in the inset of fig. for t w = 0, 
signalling a depinning transition. In the critical region, 
the behavior is power-law: P(oo,0) ~ (p* — p) 22 . It is 
important to notice that the contribution from isolated 
sites to the plateau is small, and most of the sites form- 
ing the plateau come from the infinite cluster. At p* and 
above, the interfaces are no longer constrained, the num- 
ber of active sites suffer a sudden increase and all sites 
eventually change strategy. It has been shown for several 
models 1 44 4^] that in the presence of disorder (but also 
in some homogeneous systems) , the decay to the plateau, 
P(£,0) - P(oo,0), is exponential |44|J45H at large times. 



The same behavior is found here for a non Hamiltonian 
model. In fig. we present a snapshot of the system be- 
low p < p* , showing both the empty and the pinned sites, 
as well as their strategies. This configuration is almost 
stable, many of the cooperator groups shown no longer 
change, and the small amount of active sites are either 
confined to a few regions or belong to one of the blinkers 
present. This has to be compared with the spatial chaos 
region above p*, where there is no blocking and every 
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FIG. 3. Persistence P(t, 0) for L = 100, b = 1.4 and sev- 
eral total densities p < p* as a function of time (measured in 
Monte Carlo steps). Notice that these densities are far above 
the percolation threshold. As t —* oo, all curves attain the 
p-dependent plateux P(oo,0), shown in the inset as a func- 
tion of the lattice occupation. The behavior of P(oo,0) near 
p* is power law, with an exponent aproximately 2.2. 



For densities p > p* , considering only elements that be- 
long to the infinite cluster, the persistence goes to zero, as 
is shown in fig. ^. For p = 1, the behavior is exponential, 
as shown in the inset of fig. 0. Notice, also in the inset 
(for p = 0.99), that as we dilute the lattice, only the ini- 
tial behavior is exponential. After this initial exponential 
decrease, whose length decreases as p departs from 1, the 
behavior follows a power law, P(t, 0) ~ t~ e ( p \ and as we 
approach p* from above, the plateau starts developing. 
The exponent 9(p) is the persistence exponent and has 
non trivial values, depending on the total density. For 
example, (9(0.98) ~ 3.6 and 0(0.99) ~ 5.7. 
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FIG. 4. Persistence P(t, 0) for L = 200 (empty sites) and 
500 (filled ones), b = 1.4 and several total densities p > p* as 
a function of time (measured in Monte Carlo steps). Differ- 
ently from the p < p* case, here the persistence goes to zero 
(taking into account only sites in the infinite cluster). For 
p = 1 it vanishes exponentially (see the semi-log plot in the 
inset), while near this point an initial exponential decay can 
be seen. Below p — 1, the longtime behavior follows a power 
law, unless we are too close to p* , where a crossover behavior 
to the plateux is noticed. 



stant fraction of sites in the infinite cluster that never 
change strategy, meaning that the clusters are localized. 
As we approach the deppining transition, some of the 
groups are depinned and the persistence decreases. Since 
some of the groups are still pinned, we observe a constant 
plateau that decreases as p approaches p*. Notice that 
below the transition, the number of active sites is not 
zero, but small, due to blinking sites. 




FIG. 5. Snapshot showing the sites that have not changed 
strategy after 10 J steps for L — 50 and total density 0.9 (top). 
The figure at the bottom shows, at the same time, the cooper- 
ators (empty circles) and defectors (crosses) . The black points 
are the empty sites. Those sites that are blocked will remain 
blocked forever. 

Below the critical value, the groups of cooperators are 
localized and their borders cannot move because of the 
presence of the defects. We say the groups are pinned 
by the environment. This explains why the fraction of 
cooperators is larger in this regime. As the number of 
defects decreases, the groups start to become depinned, 
the interfaces start moving and interference effects be- 
tween the groups appear, explaining the sudden increase 
in the number of active sites. From the persistence data, 
we can see that below the critical value, there is a con- 



IV. CONCLUSIONS 



One of the main features of the Prisoner's dilemma, 
responsible for its widespread use in the problem of co- 
operation, is its robustness. Here we have shown that 
the system is able to sustain cooperation even under 
the presence of lattice disorder, what was already sug- 
gested in ref. P7|| . However, what had been unnoticed 
was the dynamical transition as a function of the amount 
of quenched disorder as well as the fact that the disorder 
may enhance cooperation. Dilution changes the scenario 
presented by Nowak and May [B5| for the filled lattice in 
a dramatic way. The spatial chaos is absent when the 
disorder is above a given amount, what is reflected by 
the small number of active sites and the pinning of the 
interfaces. It must be noticed that even without disorder, 
we are not dealing with a system presenting coarsening 
as the movement of the interfaces is not ruled by surface 
tension as, e.g. in the Ising model. 

Besides presenting a dynamical transition in the pres- 
ence of disorder, dilution can even enhance the fraction 
of cooperators in the population. Indeed, in the region 
4/3 < b < 3/2, the relative density of cooperators in 
the population can reach a value almost 40% higher than 
in the undiluted case, the maximum happening at the 
transition point. 

Some questions, however, are still open. For example, 
if the condition of strong population viscosity is relaxed, 
disorder is no longer quenched and diffusion is allowed, 
what happens with the cooperator groups? Obviously, 
the results depend on the chosen rule for the diffusing ele- 
ments, and there are several biologically motivated rules. 
Work is in progress in this direction. Moreover, from the 
point of view of the study of persistence, it would be in- 
teresting to study the behavior of the P(t,t w ) above p*, 
for 1 -C t w -C t (instead of t w = 0) as well as to com- 
pare the results presented here with the site diluted Ising 
model. 
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